Skip to content

feat(google_genai): add tracing for google-genai #13650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 95 commits into
base: main
Choose a base branch
from
Open

feat(google_genai): add tracing for google-genai #13650

wants to merge 95 commits into from
+1,129 −9

Conversation

maxzhangdd
Copy link

@maxzhangdd maxzhangdd commented Jun 11, 2025 •
edited by Yun-Kim
Loading

MLOB-2932

This PR adds support for APM tracing of Google's GenAI Python SDK. Traces currently only contain UST tags as well as provider and model (LLMObs tracing of inputs/outputs and metadata will be done in a later PR).
Traced calls:

  • google.genai.models.Models.generate_content
  • google.genai.models.Models.generate_content_stream
  • google.genai.models.AsyncModels.generate_content
  • google.genai.models.AsyncModels.generate_content_stream

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

datadog-datadog-prod-us1[bot]
datadog-datadog-prod-us1 bot reviewed Jun 11, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 11, 2025 •
edited
Loading

CODEOWNERS have been resolved as:

.riot/requirements/1360370.txt                                          @DataDog/apm-python
.riot/requirements/153608c.txt                                          @DataDog/apm-python
.riot/requirements/1bc194f.txt                                          @DataDog/apm-python
.riot/requirements/d7e97af.txt                                          @DataDog/apm-python
.riot/requirements/e9955b3.txt                                          @DataDog/apm-python
ddtrace/contrib/_google_genai.py                                        @DataDog/ml-observability
ddtrace/contrib/internal/google_genai/_utils.py                         @DataDog/ml-observability
ddtrace/contrib/internal/google_genai/patch.py                          @DataDog/ml-observability
ddtrace/llmobs/_integrations/google_genai.py                            @DataDog/ml-observability
releasenotes/notes/google_genai_apm_tracing-a88d4a4dada947d6.yaml       @DataDog/apm-python
tests/contrib/google_genai/__init__.py                                  @DataDog/ml-observability
tests/contrib/google_genai/conftest.py                                  @DataDog/ml-observability
tests/contrib/google_genai/test_google_genai.py                         @DataDog/ml-observability
tests/contrib/google_genai/test_google_genai_patch.py                   @DataDog/ml-observability
tests/contrib/google_genai/utils.py                                     @DataDog/ml-observability
tests/snapshots/tests.contrib.google_genai.test_google_genai.test_google_genai_generate_content.json  @DataDog/apm-python
tests/snapshots/tests.contrib.google_genai.test_google_genai.test_google_genai_generate_content_error.json  @DataDog/apm-python
tests/snapshots/tests.contrib.google_genai.test_google_genai.test_google_genai_generate_content_stream.json  @DataDog/apm-python
tests/snapshots/tests.contrib.google_genai.test_google_genai.test_google_genai_generate_content_stream_error.json  @DataDog/apm-python
.github/CODEOWNERS                                                      @DataDog/python-guild @DataDog/apm-core-python
ddtrace/_monkey.py                                                      @DataDog/apm-core-python
ddtrace/contrib/integration_registry/registry.yaml                      @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/llmobs/_integrations/__init__.py                                @DataDog/ml-observability
ddtrace/settings/_config.py                                             @DataDog/apm-core-python
docs/integrations.rst                                                   @DataDog/python-guild
docs/spelling_wordlist.txt                                              @DataDog/python-guild
riotfile.py                                                             @DataDog/apm-python
supported_versions_output.json                                          @DataDog/apm-core-python
supported_versions_table.csv                                            @DataDog/apm-core-python
tests/llmobs/suitespec.yml                                              @DataDog/ml-observability

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 11, 2025 •
edited
Loading

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 300 ± 10 ms.

The average import time from base is: 300 ± 6 ms.

The import time difference between this PR and base is: 1.8 ± 0.4 ms.

Import time breakdown

The following import paths have shrunk:

ddtrace.auto 1.280 ms (0.42%)
ddtrace 0.657 ms (0.22%)
ddtrace.bootstrap.sitecustomize 0.623 ms (0.21%)
ddtrace.bootstrap.preload 0.623 ms (0.21%)
ddtrace.internal.remoteconfig.client 0.623 ms (0.21%)

@maxzhangdd maxzhangdd changed the title WIP feat(google_genai): [MLOB-2932] add apm tracing for google-genai Jun 13, 2025
@maxzhangdd maxzhangdd closed this Jun 13, 2025
@maxzhangdd maxzhangdd reopened this Jun 13, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jun 13, 2025 •
edited
Loading

Benchmarks

Benchmark execution time: 2025-06-24 22:05:12

Comparing candidate commit db1b516 in PR branch maxzhang/google-genai-integration with baseline commit b152e1c in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 559 metrics, 3 unstable metrics.

scenario:telemetryaddmetric-1-distribution-metric-1-times

  • 🟥 execution_time [+296.657ns; +337.182ns] or [+10.159%; +11.547%]

scenario:telemetryaddmetric-1-gauge-metric-1-times

  • 🟥 execution_time [+217.810ns; +257.854ns] or [+10.151%; +12.018%]

maxzhangdd added 23 commits June 16, 2025 12:30
@maxzhangdd
skeleton for google gen ai integration
0f7eb0f
@maxzhangdd
more skeleton, added to settings/_config.py
fc11773
@maxzhangdd
get span attributes from request
4bc8e81
@maxzhangdd
add skeleton methods to GoogleGenAI Integration
b6774bf
@maxzhangdd
incremental progress on tagging request response
245dd33
@maxzhangdd
updated tag_response
88a6346
@maxzhangdd
added llmobs_set_tags to traced_generate
dbe7406
@maxzhangdd
changed normalize_contents, progress on tag_response and tag_request
3c0bffd
@maxzhangdd
simplify code to just apm tracing, also added provider extraction
d8abc0c
@maxzhangdd
change CODEOWNERS
37cda72
@maxzhangdd
add tests
4183303
@maxzhangdd
add test for extract_provider_and_model_name_genai
95365d6
@maxzhangdd
linting, add ignore arg on snapshot
c8963b6
@maxzhangdd
fix bare except
aa3eaab
@maxzhangdd
more linting checks
f6549b4
@maxzhangdd
modified suitespec
545c3a5
@maxzhangdd
add google_genai component
a9e197b
@maxzhangdd
change google_genai component
72ae21b
@maxzhangdd
fix min_compatible_versions, changed config, changed doc
fd72fa7
@maxzhangdd
lint fix
b9d5720
@maxzhangdd
ruff
977ac43
@maxzhangdd
add supported versions
7f6e417
@maxzhangdd
lint
87952f2
Yun-Kim
Yun-Kim reviewed Jun 23, 2025
View reviewed changes
Copy link
Contributor

@Yun-Kim Yun-Kim left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments but otherwise LGTM once addressed! Great work 🎉

@maxzhangdd maxzhangdd marked this pull request as ready for review June 24, 2025 15:14
@maxzhangdd maxzhangdd requested review from a team as code owners June 24, 2025 15:14
@maxzhangdd maxzhangdd closed this Jun 24, 2025
@maxzhangdd maxzhangdd reopened this Jun 24, 2025
@Yun-Kim Yun-Kim changed the title feat(google_genai): [MLOB-2932] add apm tracing for google-genai feat(google_genai): add tracing for google-genai Jun 24, 2025
Yun-Kim
Yun-Kim approved these changes Jun 24, 2025
View reviewed changes
Copy link
Contributor

@Yun-Kim Yun-Kim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock but 3 small nits!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yun-Kim Yun-Kim Yun-Kim approved these changes

@datadog-datadog-prod-us1 datadog-datadog-prod-us1[bot] datadog-datadog-prod-us1[bot] left review comments

@ncybul ncybul ncybul left review comments

@RamyElkest RamyElkest Awaiting requested review from RamyElkest RamyElkest is a code owner automatically assigned from DataDog/python-guild

@emmettbutler emmettbutler Awaiting requested review from emmettbutler emmettbutler is a code owner automatically assigned from DataDog/apm-core-python

@tabgok tabgok Awaiting requested review from tabgok tabgok is a code owner automatically assigned from DataDog/apm-python

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@maxzhangdd @Yun-Kim @ncybul